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■ASPECTS^ OF ; £ODING ' 

< 

INTRODUCTION 
• * 

1<1* What is Coding? „ 

% • . i * 

Coding is 'a branch of information and communication 

science, Jt draws extensively upon man^diverse v 

mathematical fields-;* primarily abstract*'and 1'inear 

* • « * 

algebra, numb'er theory, probability an'd statistics, a<nd 

'combinatorial theory. If you are; a frustrated 

applications-oriented algebraist seeking a" "real world" 

putlet f<*r your knowledge > coding justmight be the. 

answer. 1 

Cbding can be thought of as a sort of reverse * . 
shorthand. By means* of shorthand one is able, byomitting 
eertain letters from wo^ds , to reduce transcription time. 
However, this saving in- timers counterbalanced by'the 
factthat one*is more likely^ to misread a shorthand word 
than an English language word. Iivcoding, information, ij 
the form of blocks of binary k-tuples, is transmitted oveV 
a nQisy channel* The noisiness of ihechannel presents 
the 'possibility that the received ^-tuple .may differ, from 
the transmitted one. However, by lengthening each k- tuple 
to be transmitted -by adding digitskTto it, channel distil 
tion is made Le-ss likely. Of course, these additional" ' 
digits increase transmission 'time.t? A central problem 
of coding, is ;to find the most effi^ent, way "of adding 
the^ so-called chedk dibits. & 

_ ' \ * " > fjl 

1. 2 The 'Coding Process 

The processes involved in coding are illustrated in 
the flow- chart in figure 1^, Tne biWary encoder receives 
the 'discrete outputs of a communications device and 
associates a binary k-tuple (k* a predetermined positive 
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Figure 1 y The coding process.* 

integen) with each of them. These outputs may be 'Tn the 
fam of human speech* , high, frequency radio waves, numerical^ 
dat^a^orC'in a host of other forms . The' k-tupies so formed 
are called rnes$ages. The channel encoder ^receives a * 
message from the binary encoder arid, by adding n-k binary 
digits to it, forms a binary n-tuple (n also a predeter- ^ > 
mined positive integer, greater than k) . The set of all * 
n-tuples formed in this manner is called a code and the. 
.n-tuples in the- code are called codewords. The channel 
is the medium of transmission (e.g., telephone lines,* 
high frequency radio links, space communication links). 
We assume tha-t the channel is noisy 0 (i .<e . , "what goes* in 
is not necessarily what comes out") . r • 

The cpding -sequence is thus*: An output enters ' the 
binary encoder wH#re it Vbecomes'" a message which in turn 
"becomes 11 a cbdeword. The channel (possibly) perturbs* 
this codeword into another binary n-tuple and transmits • 
this (possibly) perturbed^ n-tuple to thfe Jchannel decoder. 
Upon receipt of this (possibly) perturfltd n-tuple-, the 
channel decoder, which has knowledge of a\»l possible 
codewords, attempts to determine which codeword in fact 
entered the channel. The channel ..decoder then sends itj&i* • 
decision (a codeword) to the birtary decoder* which , by N 
simply reversing the procedure ,of the channel, encoder , 
determines^ the message contained in the. codeword received. 
If the channel decoder make's the correct decision, the 
message, leaving the binary encoder is identical to the 
message leaving the binary decoder. 



/ 



1.3 ; The Channel 

There are several mathematical models* for the 
'"channel. .We shall deal exclusively with* the Binary 
Symmetric Channel (BS'C) . A schematic '-diagram of this 
channel appears in' Figure 2. * 



q r 




Figure 2. The BinarV Symmetric Channel. 

With .the BSC a given bit (0 or 1) has probability q. 
m of being transmitted unaltered and each has probability 
p =» 1 • .q Of being changed into the other. The BSC 
assumes th^t errors occur randomly and independently of 
one another. We shall stimulate that p < q- (actually, to 
be considered a. "good"* BSC p has to be *oru the brder of 
1<T 5 )\ \, . . . ' 

- 1.4 Decoding * - 

i The channel decoder assumes all codewords are 

Equally likely -to have. been transmitted and makes its 
decisions* according to the principle' of maximum likelihood 

. *Aj received - n-"tfuple'r is decoded dnto that y 
codeword c w'hich dif f ers ' froon r in the 
least number of piaces.* 

e It is now time for an example. 

Example 1 . Suppose we have .a communications device 
with four outputs /a/ b, c, d, and choose k = 2 so that 

1 / 

*ln.the event of "ties" additional decoding "cri tteri a must be 
given. i • 3 
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each of them is represented by a binary 2-tuple, say 
a ^00, b'-H-lO, C++ 01, d«+ll.* Without channel encoding 
a message would have to be .transmitted through the BSC 

without error in "order to be correctly interpreted / /The 

' 2 * 

probability of Vhjis happening is*^ q . Le,t Us. see what 

happens if we add two check digits to each message/ Cy (We 

are taking n *= 4.- Precisely how -these* check digits are 

chosen is discussed later and is, of course ,> of crucial 

importance* but not for the purposes of this example.) 

We then obtain the four codewords 00(H), 100l', Oil!, 1110., 

< ' ' 

Now,* using maximum likelihood decoding, we make up ^the . 
following decoding table, - *. , 

• : ' 



0000 


1001 


0111 


1110*- ' 




0100 


''l 101 


0011 


' 1010 


> * * 


0010 


1011 


0101 


1100 




0001 


1000 


0110 


1111 





At the top of each column appear the codewords. The* 
other 4-tuples appearing in a column a^re those 4-t&Vies 
which differ from the codeword atj the top of the column 
in fewer places than they differ from the other Codewords 
(Ties have been broken by assuming an error has occurred 
in position 4.) All 16 binary 4 -.tuples appear in the 
table and if a 4-tuple r is receaved at the channel 
decoder -it is decoded 'into the codeword c at the* tap of 
its column . 

/ Have we actually increased the probability of 

correctly interpreting a message by resorting to this 

procedure of adding check digits? A codeword will be 

correctly decoded if and only if it or any 4 -tuple appear' 

ing i-n its column are received. Each of the noncodewords 

appearing In a column differs from the codeword at the 

top ill exactly* one of the positions 2, 3, or 4.- Thus if 

a given codeword is transmitted, *the probability of it 
t i 

or' a 4-tuple appearing in its column being'" received is 




+ 3pq 3 which can be shown (see Exercfse 3) to be* * •< 
greater phan (the non^encoded probability) as long as 

p. <- q.- „ . - ^ * * 

* ■* 

1 : — 

Exercises >. ' 



\. • Form a deco ding table for the code consisting of % the Tour x 
c6dewords 11000* 0011%, 100ll, 01101. ^ • 

-« « « 

For the code 'in Exercise 1, if the codeword ^ 1 000 i s,transmi tted, 
what is- ^he probability that it will be xorrectly decoded? * 



1 X. 
3 % JshOw tha>t + 3pq^ > q ? if P' $ q,. * . , « 
■ x • 

" - t ■• • ~ — ~ — ' 

1.5 ^Shannon's Theorem / r * > * 

^With regard to Example 1 another question arises, 
namely ."Cart we \io better?"" That ^s, c^n we somehow'alter 

% our code (to obtain.'a new and different code) so as to • 
increase the probability of correct decoding? In dealing 

* with this question we sh^ll fodtis upop, the* eji^cts of 

changing'k and/or. n and not Upon the actual^ # binary digits* 
in our messages and codewords* . It seems plausible that « ' 
by fixing k and, increasing ,n, say^by repeating the message 

' a sufficient number, of times, we can obtain a code with 
probability of correct decoding as -close* to 1 as wej desire. 
However, wlis« method of repetition" ha's the serious -draw- - 

^ back of greatly increasing 'transmission time** and decreasing 
the oodf rate k/n. m . Generally speaking, in forjning "good{ ! 
cpdes we. seek to maximize both the code rate a^id the^; 
probability f of correct decoding. By virtue of a •* 
.remarkable theorem we can state, categorically tha${ siich 
'codes do indeed exist"* « . 

Theorem 1 (Shannon's Fundamental/ Theorem of Codin g)* ' 
^ % Let K be the capacity* of a given BSC. Given any real 
numbers R and e, where 0 < R < K. and e > 0, there exists. 

: ; A * , \ 

*Capacity Is a positive number associated with a BSC and is a 
function of the probability p only. In fact, we have vJ* 
•K» 1 - plog 2 p - (1-p) log 2 (1-p), 5 

■ * . ft* 
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i ■ a. code with, cede rate > R and probability, of cprrect 

decoding > 1 - e . ' *^ , 

The proof of this theorem establishes existence 
. rionconstructively . This' is very -unfortunate Of orttfnate?> 
' fer it -forces lis into an in-dep"th study of coding in order 
to produce desirable codes. This is *prpci*sely what we 
initiate* in the next chapter. , . , • . 

. ' • * / * 2.'i LINEAR CODffS ^ - 

2.1 The Code Concept Refined . > " T 

Until now the only way we have of describing a coxle. is 
< as a subset of binary n-tuples. . As there is not much-one 
can say about or do %ith arbitrary s;ets* of * n-tuples , we- - 
0 shall have to restrict our concept of a^code somewhat. 

This restriction will,, however, .pay 5 great dividends .in . •» 
the quality of the results obtained. # ' .n/< 

1 • L , et ^.n ^ e the . vector sp?ce of t binary n\tuples over ' 
GF(2). The field GF(2) is v the two-element (0 and t) field^ 
^ of bina7y arithmetic, with* 0 + 0 1 -+ 1 = 0 ,. . * 

1 + 0 = 0 + 1 = i f b : 0 = -i °4 -= o-- 1 = 0, i-i = 1. Thus, 

t the elements o£ V R are n-tuples o*f zeros and bnes from 

f this .field. i: 4 * * 

r ' . ■ * / 

Ar^.(n;k) lirfear code is a k-dimensional ! 

* » . ' * ' " ,« 

subspace of V . V • - 

* • 
* In what follows the adjective linear will be omitted 

^and we shall simpJLy -%^eak of the (n,k) code C or., simply, 

"the tode;c. . \ • ' 

■ * V • 

— : » i- 

Ex eye? ses - ' 1 > * ' • 

, *" 
How many codewords are there ia^an (n,k)"code? * 

"5* Show that the three 5-tuples. (1 ,0, 1 ,0,0) , (1,0,0,1,1), (1,07!, 1,1) 
constitute a -lineaMy independent subset of V^ t J 

6. Find the^codewords In the (5,3) code which has the vectorsVin * 
Exercise 5 as a basis. ' • . * 



2.2 Hamming Weight and Hamming Distance 1 

Let x be a vector (n-tuple) in V . The* Hamming ' *- 
weight, w(x) , of x is the number of l's appearing , among* 
the coordinates of x. From the definitions of binary arid 

* vector addition we see that the only coordinate positions 
where x+y has a 1 are those positions where x or y (but, 

• not both) have a 1. Therefore 

(1) , w(x+y) < w(x) + wty). 

* _/The Hamming distance ^^(x v y) , between, two n-t.uples 
—-x-zrrfr'Y it -the 'htobje f ; r, of pdsitions in which they differ^ ^ 

Note tha^ i jgjjC^ " < _ ^''V 

(2) , d(x,y) = w(x-y) 

and d is a metric. That is 

(i) d(x,y) _> 0, with equ'ality holding if and.' 
only if x = y. 

(3) ^ (ii) d(x,y) = d(y,x). - ' ^ 
* (iii) d(x,y) < d(x,z) + d(z,yj. 

• 

Exercises 



7. Use Equation (2) to establish the fact that Hamming distance sat- 
isfies Equations ^(i-)-{i i i) in (3) above, so that it is a metric. 



2.3 Decoding Revisited 

^ Viewing V n as a group (with respect^ to n-tuple addic- 
tion), and an (n t k)^C<5^e C as .a st$>group, we may J^yrm the 
cpset decomposition of "V with- respect to C . 

V 

(4> V - u (a-+C), 1 * * * (disjoint). ' \ 

where ^the* cose t repre$enta.tives a.^ are chosen to be 

n-tupl.es of minimum weight in their cosets. Letting 
* » 

A f {a i > and C - {c i } > it fpllows directly from Equation 
(4) that * * 7 . 
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' 2 k 

j ( $ ) ' fc V n = U (VA). 1 (disjoint) ■ 

• i=1 . • . ^ 

.In tjsrms of Hamming distance, maximum likelihood 
decotfing^ re,ads as follows: A received n-tuple r is 
decoded intp that codeword which minimizes {d(c ,r)/ceC} . 
A£ a consequence of Equation (5), for any received n-tuple 
r thfere exists a unique codeword c. and an a, in A such 
that r = c. + a v . . Then for any c in C we have , „ 

(6) s c-r = c - (Cj+a k ) = a k + (c-c^) e a k + C ~> • 

: • s => d(c,r) = wfc-r) > w(a k ) = w(r-Cj) = 0(Cj ,r)* 

1 . ' \ - ' 

| which N tells us that r* should be decoded as Cy' 

From the result above we can deduce that the rows of * 
the decoding table for C are the cosets appearing in Equa- 
tio'n (4). This observation allows for the simple 
construction of the table. The algorithm is as follows: 
A§ is standard, the first row consists of the codewords f * 
themselves. Of the remaining n --tuples, one of minimum 
weight- is chosen and placed under, the zero codeword . ^Each 
of the remaining n-tuples in the second row is the sum of 
the codeword immediately above it and this minimum-weight 
n-tuple (see the 1 decoding 'table in Example 1). Now once 
\ fc agairi" an n-tuple of minimum weight is Selected Jfrom the 
remaining n-tuples, placed under the zero codeword, and 
the third row* is filled out as was the second. The 
procedure is'repeated until all n-tuples <are exhausted. 

* 

Exerc i ses 

8! Form a decoding table for the 'Ts, 3) 1 code of Exercise 6. 



2.4 The, Generator 'and Parity-check Matrices 

A kxn matrix whose rows are basis vectors for an 
(n,k^ code C* is called a generator ^matrix. Writing down 



*Note that a^ = -a^. 



1.0 



such ajpatrix-is actually ja compact way of specifying a 

code,* for knowing its k rows enables us to determine all 

k ** 
the 2 codewords in the co'd^. (This is a significant 

observation, for there are codes in use with .k greater 

than fifty.) 




The subspace of -V n orthogo\al>o-C is denoted by C x 
• and called the dual code of C. That is 

C 1 =*{xeV n |x-y = 6 for all yeO) , 

where x-y is t*he usual dot product of n-tuples\ Since the 
^dimensionof C 1 >is*n-k, C 1 is an (n,n-k) code.*\ 

Let H be an (n-k) x'n generator matrix 'for C 1 . Then 

(7) xeC if 'and only if xH T = 0. s 

Letting x = (x x , x 2 ,*..., x r ) , and denoting by h. . the 
entry in (the ith,row and jth column of H, we may rewrite* 
Equation (7) equivalently as 

- n 

(8) xeC if and only* if £ x.h. . =0 for i ■ i; .... n-k. 

Since we are working .over' GF(2) , Equation (8) says x^s, a 

codeword if and only 'if the 'number of integers j for^which 

both x. and h.v are 1 is 'even for each i = 1, . n-k. - 

J a j . 

Fdr thTS reason we call H> a, parity - check matrix for C. 

Suppose G is a generator, matrix for ait (n,k)'code C. 
The reduced echelon ^form. of G also serves as a generator 
jnatrix for C. By permuting certain column's of the 
reduced echelon form of G we obtain a kxn matrix of the 
form G f » [I k P] , where* I k is the k x*k ident ity matrix and 
P is a kx.(n-k) matrix Ccall the permutation involved p) .' 
The matrix G ! can be thought of as a generator matrix for ' 
an (n;k) code C ! . It can easily be verified that the rank 



t *A proof that dim C x » n-k may be based upon the observation 
that C x can be Identified as the solution space of a system of k 
Hnearly independent equations in n unknowns. 



nj-k matrix H ! = [P T I n . k ] satisfies the* equation 
G'H ,T = 0 ' 

' from which we conclude that H f is^a parity-check matrix. 



for C; . Applying p"* 1 to the columns of H» will produce* 
a .parity-check matrix H for C. 

Example 2 . Th£ matrix 

'fioiVo' 
g = icon 

' [4 ,0 1 1 lj 
4 

is a generator matrix for the (5 % ,3) code of Exercis-e 6. 

The reduced echelon form of G is 

fl 0 0 0 0] 
E = 0 0 1 0 0 ,, 
(o 0 0 1 lj 

By -applying the permutation 

P = (243) 

(written- in cycle form) to t^he columns of E, we obtain the 
matrix 



G 1 - 



1 0 0 0 0 
0 10 0 0 
0 0 10 1 



which is of 'the form [LP). Then 



H' =JP'I 2 ] = 
and applying • 



0 0 0 1 0 
0 0 1 .0-1 



P f - (234) 

to the columns of this matrix we get 

'0 1 0 0 0) 
,0 0 0 1 lj ' , . 

\ 



i 



i' H 



15 



10 



\ 



^2.S Systematic Codfes # • 

Codes like the cpde C 1 of the preceding section/ 
irthich have a v generator matrix of the^form'G 1 , are called 
systematic codes. The^ study of systematic codes is 
-greatly facilitated by , the simple natur„e of their generator 
matrices. For let r^, . .\ , r£ be the tows of G 1 . Then 
xeC* if and only if 'there exist scalars (binary digits) 



a^ such that 

x = Ta.f 1 . . 
k i i. * 



Upon expanding this sum we get 



V a k+l> a n ) 



and 



xeC, il and* only if; x = (a, , . 

k * , * 
(9) where a. - J a.g». . for j » k+1, n 
\. *i=l 1 ^ 

tgj. the row i-column j 'entry in G 1 



, Thuj- the first k coordinates (the message digits) of 
%l codeword in.C* can. b*e chosen arbitrarily while 'the 
remaining n-k coordinates (\he check digits) are linear . 
combinations of. these messageiiigits . . ' ~ 

' Example 5 , If C - (1 , 0 ,1 ^4)05) • is .to. be a codeword 
in the -systematic code C generated by the matrix G-' of 
Example 2, then we must have \ ^ 

°4 = C l8'l4 + C 28'24 + c 38*3* = 1-0 + 6-0 + 1-0 = 0. 
.V C 5 = C l*Sl5 + C 2g25 +C 3g35 = 1,0 + 0 " 0+1,1 = K 



Exercises «* . • . * 

9. Wh^t .relationship exists between £he codewords of C and T*? 

10. "Find the matrices G', H' , and H for the (6,3) cfide with 
generator matrix 

0 1 ll 10' 
f 0 1 0 1 1 , 
0 0 0 1 1 I 



faERJC 



*5 Ic 
UL 



4? 



li 



1-1*. Find the check digits for the codeword with message digits 

lii in the (6,3) code of Exercise 10. 

t 
\ 



4 5. ERROR CORRECTION 

I * 

5*1 A Criterion for Code Quality 

Suppose "that the channel is sufficiently reliable 
for the channel decoder to assume that at most t»errors 
(i.e., t alterations, *t a positive integer) Iran occur in 
a transmitted codeword: Can we then be one hundred 
percent certain that the decoder will decode correctly? 
If the answer is yes, for every codeword in the code,. we 
say the code is t error-correcting; 'More precisely, £ 
.code »C is t error-correcting , if the .closed balls 

{S(c,t); ceC},- 



where 



5(c,t) * (oceV n /d(x,c) < t}, 



are pairwise disjoint. * " ' 

* ■»'•> * ~* 

In view of .the f act* th'at w(x) » d(x,0), * ' 1 

d$x,y) = w(xry), and a code is a subspace (so £hat*x-y is* 
i-n the code whenever x and y are) , we may conclude that W, 

minimum weight of all nonzero coofevfrjjfcxls , is equal to 
D, .the minimum distance between different codewords. It> 
seems intuitively clear that for_a code to be t error- 
correcting, the codewords have "to be !t suff iciently far 
apart. Just how far apart is revealed in the* next -theorem 

Theorem 2 . A code, is t error-correcting if'D ^> it + T. 

Proof: Suppose not. Then there exist two codewords, 
c 1 and c 2 , /such that the closed balls 5(ci,t) arid'3*(c2,t) 
have an n-tuple r call it r, in common. Then 

(10) d(c x ,c 2 ) < d(c lf r) + d(r,c 2 ) < t+t < 2t + 1. 

But^ ' 

> ^ 12 



(11) ' ' d(c 1 ,c 2 ) = W(c 1 -c 2 ) >" ft « D > 2t % + 1, 

^so.wei have^arrived at a contradiction and the theorem is 
proven. *- » 

1. i 

Exerc ises 

• — ♦ 

Y2. Establ ish % the converse of Theorem 2. That is, show a code is 
t error-correcting only if 0 £2t + 1. . * 

13. Comment 'on the .error-correcting capabilities of the code in 
Exercise &. v I 

' ' . *■ 

3, .2 Error Correction and the Parity-check. Matrix - 

An extremely elegant and simple characterization of 
t error-correcting codes may be expressed *in terms of the 
parity -"check, matrix., " . 

Theorem- 3. Let H be a parity-check matrix for a code C. 
"If every:; subset of 2t columns* of H is linearly ^independent , 
then G 1 is terror-correcting. \, 

ThisWesult is an immediate consequence of Theorem 2 
*and the following ^lemma . " 



Lemma 1. If C has a codeword c of weight w, then some w 
.columns of H are linearly dependent. 

- ;}Propf: Let c = (c^-/.., c R ) . Now w(c)'= w means 
'that exactly w. of the {c^ are 1 (which we may assume to 
l?e> without loss of generality, coordinates.* 1 , 2, . .., w) . 
Now c e C means . . 

(12.) cH^ = 0. * , ' 

J)enotirig/the columns of H by h ± , h R , Equation (12) 

may be rewritten equi.vale.ntly as ' 



v 1 

or, using the^alues of the ,c if 



13 



/ (14) * £h. = 0 



1" 



Equation (14) says the first w columns* ©f H are linearly 
dependent and the lemma is proveji. % > * 

5 . 5 Hamming Codes , v '. ^ , 

We put Theorem 3 to immediate use;* Let m be a ' 
positive integer and let H be 'the m * (2*1-1) matrix whose 
•.columns are She binary representations of the integers 
X, 2 m -l respectively. For m = 3, 



H = 



0 0 0 1 1 1 1 

0 110 0 11 

1 0 To i-o 1 



' Since H contains the m columns* of the identity matrix I 
v u — j * m ' 

ranK H : - m and H can, serve as a parity-check .matrix for a 

(2 -1,2 -1-m) code C, a type of code referred to as a 

Hamming code. ^ Since two nonzero binary n-tuples are 

-linearly dependent if and only if they- afe^lentical , 

eve^ry pair of columns of -H are linearly independent. 

Hence, by virtue of Theorem 3, Hamming codes qve single 

ervovHCOvvecting . 

• Moreover, with the aid of the matrix/H the decoder 
can easily, correct any single error. For suppose a single 
erroij occurs (say in position i) in tjie transmittecf code- 
word <c in C,- so that the channel .decoder receives an n-tuple 
r' wMch ^differs frqn?conly in position i. Then, since 1 
cH T » 0, . - , : ' 

f rH T = (r-c)H T * (0,~ U.\ ' 0, ...0)H T *= h., / ' 

(the ith cblumn of H, i.e., the binary representation of i), 
thus enabling the decoder to determine the position of the 
error. 

!_3 ! , ; - 

Exercises 

14. Find a parity-check matrix for the (.15,10 Hamming code. 

14 . 



15. Find a generator matrix for the Hamming code/ 

_ * r 

~tfn — Wcrrkrng-with tfie (Tv^rfWrnrnfrrg^rode and " assu ming no i rore~tfran" 

one error occurs during transmission , >vhat codeword was 

transmitted if (0,0,1,1,1,1,1) was received? 

. » . 

3.4 Perfect Codes * 



Recall thait a code «is t error-correcting if the' * 
closed balls {5(<:,t)} of radius t with centers at the code- 
words are |>airwise disjoints If, furthermore, these £alls 
fill the*space (i.e., their union is all of V ) , the code 
is said to be perfect.- To show that llamming codes ^re 
perfect, we may use the following counting argument. Let 
n ■ 2 m -l and let C be the (n,n-m) Hamming code. Now 
#V_ « 2 n . Each of the- closed balls S(c,l) contains n+X 
n^tuples. Since #C ^ 2 k , where k = 2 m -l-m - n-nt, and the* 
balls are paixwise disjoint, the union 6f the balls con* 
tains 2 n " m fn+l) n-tuples. A little arithmetic will show 
that 2 n ^ 2 n V a (n+l) when h - 2 m -l. % 

3,5 The Baseball Pool 'Problem 

'Hamming codes can be used to, supply a simple (alb.eit 
partial) solution* to a rather intriguing problem in m . 
combinatories, the, Baseball. Pool Problem: ."On- a- given day 
n^baseball games are* to be played.- If . a single, bet* is 
defined as picking* the winner in each of the n ganfes, what 
is the minimum riumber of bets-one has "to make to guarantee 
choosing at'ieast n-1 winners? 11 Clearly,^ 11 " 1 bets .are 

"sufficient to guarantee at least *n-l winners. We* solyfc * 2 
the .problem for n of the form Z m -1 By* denoting a homeV,* 
team victory by 1 and a home team defeat by JO, each of r 
the 2 n possible bets can be associated with a binary,' 

-n-tuple (i;e«, a vector in V Noting that Hamming codes 
are single' error- correcting .and perfect , the only- bets we 
need A place to guarantee ours elve> at least n-1 wfcniiers 
are the 2 n " m ^ n- tuples v the (n,n^m) Hamming codcT 'TKus, 
when n ■ S'Vltf* we 'have reduced the sufficiency number from- 
2 n - l - t o 2, n ^. % ' " % -\J ' • • 1 



r 



Exercises 



*n-m 



17* Show that 2* is indeed a minimum. 
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5. AhlSwERS TO EXERCISES 



1 



. 11000 


00110 

— - • * 


10011 


- 01101 




11001 


00110 


jooio 


01100 




11010 


00100 sS 


,iiooi 


01111 




« * 11 100 


00010 


10111. 


01001 


r 


* 16000 


"tniio , 


* 1 1 01 1 


00101 


uioob 


10110 


00011 


11101 




11110 


, 00000 


01011 
11111 


10101 




01010 


, 10100 


00001 , 
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Tne 5"*tup1es below the codewords and above the dotted -line 
differ from the codeword at the tofTTiTone frcsT^orT and" are 
uniquely decodeable. Those 5-tuples below the dotted line differ 
in two positions from two or more codewords and are not uniquely . 
decodeable us ing> only^the principle of maximum likelihood. 

* . ■ t 

Z.* ] 1000* will be decoded correctly if and only if a 5" tuple in ijts 
column is received.^ One of these 5-tuples'differs- from 11000 
in*2ero positions, five differj in one position, and two differ 
in two positions. Hence the probability of correct decoding is 
jq + 5pq o+ 2pV. 

* ■ -' 

3. Since p < q, ,1/2 < ,q <]'. Then 

q k + 3pq 3 '- q 2 » q* + 3(1-q)q 3 v - q 2 » -2q k + 3q? - q 2 
<* ' * - 

* q 2 (-2q 2 *3qH ) » 2q 2 (l/2-q) (q-1 ) > 0. 

i * k • * 

**. Since there are only two scalars, there are 2 codewords in an 

« (n>K) code. 



5. For three -nonzero vectors in V^7to be 1 i nearly ^dependent, we must 
have either. » ♦ 

(i) two or more of the vectors are equal, or 
(ii) one of the vectors is^the sum of the other two. 
Since neither, of these condtions Is true for the three vectors 
giyen^the vectors are linearly, independent. - 

6. * We find the seven nonzero codewords by forming all' possible sums 

of the basis vectors taken 1., 2, and 3 at a tifte. The codewords 
are 00000, 10100, 10011 , 10111, 00100, >0001 1 , 00111, 10000. ^ 

7. As (i) and (ii) of Equ^^o^T^ are obvious, we shall only 
prove (iii) * * . * 

d(x,y) » tf(x-y) » w(x-z+z-y) < w(x-z) -+ w(z-y) « d(x,z) ^ d(z,y). 

* * » * 

As* In Exercise "1, the table below is not the only possible one. 



000^)0 


10100 


1001 1 


10111 


00100 


0001 1. 


00111 


10000 


01000 


11000 


1 1 CM 1 


inn 


01100 


01011 


oi fn 


v 11000 


00010 


10110 


,10001 


10.101 


00110 


oooo i 


* 00101 


10010 


01010 


11110 


11001 


11101 


01110 


01001 


01101 


11010 



The codewords of C 1 can be Obtained by applying p to the code- 



words of C. 



The reduced echelon form o^ G is 
V E 



i o 1 o i # 

0 110 0 1 
0 0 0 1 1 1 



By applying the permutation p = (3*») tQ^the^ columns of E, we 
obtain 

1 0 0 1 1 f 
0* 1 0 1 0< 1 
0 0 10 11^ 

which Js of the form [l-P], Then 



H 1 » [P 1 ^] = 



-I 



i ro i o o 

10 10 10 

1 .1 i o o ij 



and applying p (« p) to the columns of this matrix we get 



1 1 1 0 0 0 
10 0 110 
1 1 0' 1 0 1 




\ 



Let C be the systematic code generated by^the matrix G' of 
Exercise 10. Then c'~ (JJJ.cj,,^) is the codeword of C 
with message digits 111. Furthermore, 

C V S c 1 9 1if + c 2 9 2if + c 3 9 3if - M + 1*1 +V0 - 0 
. c 5 * c 1 9 15 + c 2 9 25 + c 3 9 35 -1'1 +1*0+1*1 -0 
c 1 9 16 + c 2926. + c 3 9 36 a ' M + 1,1 +1-1-1 
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m ThussC 1 * (1,1,1,0,0,1) and the codewojAd In C with message 
digits 111 is, using the idea in fxercfise 9 and the fait tha't 
p" 1 - (3*UU,0,.1,0J). 



12. Suppose D < 2t + L. Then "there exist two codewords * and y 
• such that d{x,y) < 2t +"1 . Let r e bt obtained from x by 
changing m of the digits in which x and y differ, where 
id(x,y) < m < t . - Then, f • 

1 d(x,r) - m< t* and <i(y,r) < id(x,y) <_ t * 

the code is not t error-correcting. 



17. • Suppose A is a collection of bets (i.e.* a subset of V ) which 
% * n 

guarantee at least n-1 winners. Then V = (j {S(a,l)} =*> 

n aeA 



U {S(a,l)} 
aeA 



£ I ns(a,1)} 
aeA 



(#4)(n+1) *#A > i n 



13. 



1*. 



•The minimum weight of ^he code is 1 so that it has extremely 
poor error-correcting capabilities. In fact a single error in 
the codeword '00000'wi 1 1 produce the codeword ,00100. 



15. 



00000001 1 1. 1*1 1 1 
0 0 0 1 1 1 1 0 0 0 0 \\\ 1 
011001100 1" 1001 
1010 1010101010 



As a parity-check jnatrix foe a code C may be thought of as a t 
generator matrjx for the code C" 1 , and a partty-check matrix for 
C x regarded as a generator matrix for all we need-do is apply 
the same technique as in Example 2.with^the roles of G and H 
reversed. Doing so, we ,find ! * 



1 11 0 0 0 0 

10 0 110 0 

0 \fb 1 0 1 0 

1 1 0 TO 0 I 



is. a generator matrix for the* 
{7 M Hamming code. / 



16. rH' 



(qouiii) 



.24 



0 0 1 

0 1 0 

0 1 1 

1 0 0 
10 1-| 
1 1 0 

1 '* y 



£* (0U), which is the binary "representation 
of 3- Thus ,there is, a single 
error in position 3 and the code- * 
word transmitted was (0,0,0,1,1,1,1). 



r 
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1. 



INTRODUCTION 



Algebraic coding theory originated in the late 
1940»s in the attempt^ to solve the problem of transmuting 
an' electronic message^ through a noisy channel. From us 
beginning, as a hybrid of algebraic and probabilistic 
results, the theory has developed, while shedding light* 
.on the original engineering probiem, to. the«point where, 
if is being applied in other . areas of mathematics (e.g., 
group theory and combinatorics^ . 

This -module will provide you w&ith a* brief introduction 
to algebraic coding theory, via an example: we will con- 
struct a doubler error correcting code. Only elementary 
algebraic techniques will be used. The prerequisites 
are an elementary linear algebra course and some facility 
for manipulating polynomials . * . 

If, after ^reading this .module, you would like to 
learn more algebraic coding theory, then TJie Theory-v>f 
Error Correcting Codes I and II by F.J\ MacWUliams and 
N.J. A. Sioanc is the book to- see. 



\ 



THE. PROBLEM* 



The following simple model is- appropriate, for the 
study of many communication problem!. 



transmitter * 


Channel 

— 


I 

Receiver , 


* 



for example, the model could represent a satellite 
transmitting radio signals to a station on earth or the 
transmission of telephone signals along a cable. For 
our purposes we assume that the transmitter can produce 
and send two sypfbpls, 0 and 1, along the channel to the 
receiver. In/practice all channels a/e noisy; i.e., 
occasional]/, a 1 is transmitted and a 0 is received, or 
a 0 is toransnixtted and a 1 is received. We assume that 
there is a fixed nonzero probability, p, of incorrect 
transmission. ^The probability of correct .transmission' is 
q - 1-p. .Such a channel is called a binary symmetric 
, channel . • 

q 




Let's be spe'cific. Suppose you transmit a message, 



m l m 2 ' 



•m 



l 15 



, consisting of a sequence of fifteen 



15 



symbols chosen^from {0,1} through the channel to the 
receiver. If r ~ r i r 2 * ' r l5 is rec eived can the 
transmitted message be recovered by the receiver? 
Since p > 0, r could differ from m in up to fifteen 
places. From th* receiver's point of view'any one of 2 
•* possible messages could have-' been, transmitted! However, 
.if p is close^ to zero (a reliable channel) he or she 
would not expect many errors inl transmission ; i.et, the 

message-should -be Tie^cr- t*he receiVrtTTTwd in the" sehse^ 

tha^ they do not differ in many places. Throughout this 
discussion we will assume p is so close to zero that more 
than r two errdrs in transmission a^re unlikely. (If you 
know some probability: the probability of two, or fewer 




- For exaij^],;;^ 'ihS "ieS§age£^-* : ; 

is likely , -to b£ -a'mon^-ilfe?;^ ^ •! 

fewe'r^-Sn -Even 



P.rior .to n^is^^^i^^%'.4)iio{^d- harce^c6ril£gf ee$|=3ftaf"' 
only" onewmessag.e wi t nV^b/^iD^^ fo.r 



^probably "a. single rShehi'Tn 'the thi'-rd bit, 

(iij* that the. message was^fprobarbly) 

m =• llOllHllllll-11. * 

Tfce problem: Can a set, C, of message words be*" 
# chosen from the set of binary fifteen-tuples so that 
the occurrence of two or fewer errors in transmission can 
be detected and correqted'by the receiver? "We*wilk refer 
.to such £*set Us a code and*to its elements as codewords. 

Some obvious .choices .for C are {000000000000000}, 

{ijimimuiii} arid {oooo.oooooooooqo;iiiiiiiniiiiii}r- 

But, while these choices for* C , satisfy our error , jietec- ' 
tion and correction* requirements , they do not enable us,! 
to transmit much information. , Ideally .C should contain^ 
as riSny codewords as possible.; However, since there are 
2 2 »j-l non-empty' subsets of binary fifteen-tuples -(more ! 
than a billion)^dne cannot rummage through them at 
--random- looking- -for a^ Lar-ge-oode-r— 'We-mu'st^rest-r-iet-otir— 
attention <*© sets of binary fifteen-tuples which possess - 
some sort of regular structure. Our approach is algebraic. 




3. CODEWORDS AS~ VECTORS 



-/;;We-3"enbte the set of . binary r ii f teen- tuples by Z~ 15 



^ 1S the field, with^two et&inients" 0 and \ , a:nd 

'/ pj).£rations^\ftJand , --giv*o:-j>y the" table below. 



• - - TAfcLE V. 

" Add it ion . and ~Mul t ipl icarrtoaVi*^ 



' + / 


0 


1~~ 




0 


1 


: "- 0. t 


°* 




% • 

0 




0 


''I 


i 


" 0 

t" r ■■ 


1 


0 ' 


1 



t - wise 



- Us^ng the,' addition of 2 2 to add elements of component- 
11011* 10010 > 1101 

o i i o«ro i i i i o i 0'i i 



i o i hi n o i i o i i o 



enables ys to view^Z^ 5 as a vectdr space of dimension- 
, 15 over (the finite field) 1^ . This obsepvatipn is useful 
for the following reasons. * ^X^^^ 

^ . • (*) Tne introduction of errors by the channel can 
,/T>e despribed algebraicajly . If m is transmitted and r 
' is received, then we can write -» 



(1) 



m + e f 



where. e'= e^e^'-e^ is called the error word and is 
defined as follows: 



if r. = m. 
i i 

if r. j~n~ 



Notice that .Equation (1) is equivalent to 
(2) r + e m 
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since e + e » 0 (remember, 1 + 1 o) . Thus,, finding e^ 
and adding to r yields the transmitted message. 

* * o ' t 

Problem 1. Compute e if m = 101010101010101 and 
r * 101010111010100.. 

(ii) Subspaces -of Z 2 15 *are obvious' candidates to 
server**, structured sets* of message words — codes. £ 
Better yeY , they are easy to construct! We can take 

C E Z 2 ijo be the set of solutions to the equation 

(3) Hm* * 0; 

where H is some*- n x lSTbinary matrix and m 1 is the 
> transpose of m = m 1 m 2 ...m 15 . The matrix H is called^he 
parity -check matrix of the code C it determines. (In 
the literature; H is called a Hamming matrix ,^and C a 
Hamming code*) 

(iii) The receiver can use the algebraic descrip- 
tions of C and e to advantage. It follows from (2) and 
(3) that 

Hr* = H(m+e) t * HO^ + e*) » W + He t = He*. 

j Case 1. Hr* t 0. Thi$ means that an error (or 

errors) occurred in' transmission since r^C. More- 
I ovfer the e<rror-word is among the solutions to tjie 
nonhomogeneous equation He t = Hr t . 

'Case' 2. Hr t = 0. This means that r is a codeword. 
Either transmission was error-fre,e or the., error- . 
word satisfies He* =' 0; i.e., the error-word is a 
codeword \[ % 

These remarks illustrate ± he imp ortance .of the 
vector Hr to fheerfor detection anil correction process. 
'We will refer to>41r~- t as the syndrome of r and denote it— 



4. A SINGLE-ERROR CORRECTING CODE 



To' use the observations in Section 3 we need to 
specify a parity-check matrix. Let f s try 



H, 



11111111111110 0 
101010101010010 
0 10101010101001 



The code, Cj , is the set of ^solutions to H^m t = 0 which 
is equivalent to the system: 

m l + m 2 + m 3 + ^ + m 5> m 6 + m 7 + m 8 + m 9 + m l0 + m ll + m l2 " m 13 



+ m c 



+ m 



11 



"14 



+ m, 



+ m, 



+ m, 



l 8- 



+ m 



10 



in 



12 



U 15- 



These equations are called the parity-check equations of 
C r As written, they imply that jn, , m, , • • • ,m. 0 may be 
chosen freely from 0,1 as long as m 13> m 14 , and m 15 are 
chosen t.o be the appropriations. (You might, think of v 
m 1 ,m 2 ,- • • ,m 12 as information* bits and nu 3 , m 14 ' f *and m 15 
as check bits.) This observation *(or the number of 
columns of ^ minus the row rank'of'Hj) implies that the * 
dimension. of C ± is 12 and |C 1 I = 2 12 . 

; ♦ j 

Problem 2. List a few of "the codewords. 

With this code you can transmit a lot of messages. I 
How does the receiver fare? Let's suppose W transmitted 
m and he .received r = lllllllll'liaoOO. ' [ „ 

0 V at least one error occurred in trans - 



Since H^r = 



V:: 



mission and the error-word is among the solutions to 
fll ' ' < 



H,e 4 



system 



, which is equivalent to. the nonhomogeneous 
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C l + 6 2 + C 3 + K + e 5 + e 6 + e 7 + H * H + \0 + e ll + e l2 + e l3 s 1 



+ e; 



+ e F 



+ e 



7 



+ e 



11 



+ e l4 ^ 0 



' + e. 



+ e. 



+ e, 



+ e. 



+e i2 +e l5 ttl - 



6 ' c 8 * T c 10 

The receiver might solve this system and look for error- ' 
words in the solution set whi c ch have l's in only one or 
two positions. A shortcoming of this* approach i# that 
after solving the system the receiver must examine 2 12 
words^. This is bound to be time consuming*~(expensive) ! 

A more efficient approach' to error identification is 

V 



tp view the syndrome of r, 

\ t 
tion of the columns of H, : 



t 

H^e , as a linear combina* 



+ e. 



+ e. 



+ e 



14 



+ e r 



+ e 



1 


"1" 




'1" 




T 




"1" 


+ " e 4 


0 
1 




'1 

0 


: e 6 


0 
1 

u * 


+ e 7 


1 
0 



+ e 



10 



"1" 




X 




"i" 




"1" 


0 

1 




1 

_0_ 




0 

1 


+ e 13 


0 
0 



15- 



ft) 
o 



The error-words with a'single.l are easy* to find: the.* 1 ' 
must 'occur in a position, corresponding to a column xfrichjfr 
equal, to the syndrome. All such e.rror : wofl|$, and the 4 ^ 4 ' 
porresponding^fessages, are listed in the f on^w^^array , 



niiiiiiiiioooo, + 



010000000000000 

000100000000000 

0 4 00001000000Q00 

OOOOODOIOOOOOO^ 

000000000100000 

000000000001000, 



'lOllllllMlOOOO' 
111011111110000 
111110111110000 

imiiioiyy)ooo 

111111111010000 
111H1111111000 



V 



which illustrates the receiver's problem with this code. 
He has no way of knowing which one of the six equally 
likely error-^words actually occurred. 



Problem $. If r = 111111111110000 and two errors 
occurred in transmission, what was m? 



To avoid this difficulty the columns of the 'parity- 
check matrix should be distinct.' This is impossible 
using a* 3xi5 parity-check matrix since only eight binary 
three-tuples exist. Let's? try. a 4 x 15 parity-check matrix, 
say 

"0 0 Q/d^O, 0 t) 1M 111111" 
0 0D 1 11100001111 



H n = 



0 1 1O 0 1 1 0 0 1 1 0 0 1 1 

1 0 1 0 L 0 1 0 1 0 1 0*1 0 1 



.Problem 4. ,Why is it a bad idea to have a column of 
0's in a parity-check matrix?' 



Tte "dimension of C 2 is eleven so there are 2 



11 



codewords; fewer than in , but the receiver can now 
* detect arid correct any jingle error in tranSmii 
Indeed, a single error occurs in the ith posit- 
only if, 

t .. t 

c. , 



* & v*cui nun a 

'missi'ony/ 
;i t ion 'At , and A 



.s = H^r* = H 2 e Z 



where. c i denotes the ith column of H 7 . 

Example 1. If r = 110000111111101, then transmission 
was incorrect since » 



* 0. 



Assuming a single error occurred, it was in the tenth"" 
position sirtce the syndrome is the tenth column of H, 



jn = llOOOOlllliilOl + 000000000100000* 

* iiooboiiioinoi. . ' ' ' 

Note that each column of H 2 can be viewed as^the 
binary ^representation* of the column number. Thus 
Tl-T ' *~ " ' . 0 

implies the error (assuming- a single e^rror) 

/** \ •* 

-occurred in the tenth position. The receiver simply 
changes the tenth bit of r to recover the message. 



Hgr 1 . = 



.^^^n 5. ►Show that if r - 111100101001101, then* 
transmission was incorrect. Assume a single error 
occurred** ;and recover the message. Show that more than 
one, double t err or *:an give the same syridromeNas r. 



5. . A- DOUBLE -ERROR > CORRECT ING CODE 

Are ajid C 2 useful on double errors? If two 
e*rrors occur in transmission, "say in the itji and jth 
pol^tipw, then 



<4. s = H-.r* = H-e* * c" + c! 
^ (i) Detection? x^nce the column!? of H 2 are 1 ' dis tine t 



on? N^nc 

c i .+ W^-f 0, ^/hfch implies transmission 
w§s incorrect. ' • 



and iionzero/ s ■« 



(ii*)^ Correction? Jsloj: hardly! Xhe columns * corre- 
sponding to the error locations are not uniquely determined 
by the syndrome (see Problem, 5).. This is because 
c- 



: t f c i = s is one (vector) .equation in two unknowns. 

.pother equation in c^and Cj would be helpful, 
we cpntinue to think- of parity-check matrices as* matr: 
of columns .then an 8x15 matrix of the form 



where c m is the mth column of H 2 and f(c m ) is some c , 
provides a second .equation in c. and«c.. Specifically, 





+ 






". S JL 


f(c.) 


* 


f(Cj) 




. S 2. 



implies t 
(4) 



C i + 



«(c.) * f(Cj) = s 2 , 



where the eight -component syndrome H 3 r t is ' wrijten, as two 
four-componenr syndromes. £ 

Whether (4) can be soared uniquely for c.and. c. 
depends on how the function f is defined. Given our^ 
ability j;o add columns, and multiply them by 0 or 1, about 
the only algebraic choice for f is linear: 

f K\ = bc m + c k> • 



where b e.Z^ and is a four^-component binary column. 
With- this definition of f, the second equation in (4) is 

\. 

(be. ♦ c k ) ♦ (bey ♦ c k ) =-3 2 , ' 



M c i * c j\* <v + c k ) 



'2* 



b(c. + c.) « £ti 



i V 1 ] J 2 



Either chpice of b leads to a xedunSant second equation 
i + ?5 = s 2 or 0 = s 2^ ajitfs provides no help in .deter- 



mining c^ and Cj 



,2'. 



\ Something nonljnear is required. Maybe f(c ) = (c )*\ 
wi-J.1 do, if V«er can invent a multiplication procedure for 
four-component columns. The key is to^ associate each / Y\ 

four-component column with, a binary „polynomial*of degree 
three or lesi. This i4entif icatign process 



10. 



..A 



= 1 0 1 1 ^ x J + x + 1 



preserves, column addition. 

0110 + 1011 = 1101 x 3 + x 2 + 1 = (x 2 + x) + (x^+ x + 1) , 

\ 

and makes column multiplication possible 

„•* 

< Gono) • (noli) 

(5) 



(x 2 + x)(x 3 + x'+ if 



x 5 + x 4 4 x 3 + x 2 + x 2 + X 



■ x 5 +- x 4 + x 3 + x 111010, 

although closure' is not guaranteed, as you can see. This 
deficiency is not fatal. We can identify x S + x 4 + x 3 + x 
with a polynomial of degree at most three, namely itp* 1 *. 
remainder upon division by a (previously agreed upon)' ^ 
fourth degree polynomial, say p(x) - x 4 + x 3 + 1. "Since 



x 4 + x 3 ** + 1 



X * t 

~~5 4 3 

x + x + l x +,x 

" 5 4 

X + X + X 



5 4 3 3 
x .+ x + x + xis identified with x . We denote this 

identification, by x 5 + x 4 + x 2 + x = x 3 and say »\ » 

5 ' 4 3 - " . ' ' 3 , -4 3 

x + x + x „ + *x is congruent to x/ modulo x + x + 1. 

It follows from (5), and (6) that 

«*■ 

(0110) • (1011) = 1000; i ' 

that is . 



Problem 6. Which column of H 0 serves as the 
multiplicative identity under the multiplication just 
introduced? Show that c 12 • c 2< " Cj and • c 12 = c 1Q . 

Although we will not prove it, the columns of H 2 , 
togethej with the zero column, form a finite field with 
respect to the addition and multiplication we have just 
-introduced: Less precisely, we can manipulate the 
columns algebraically as^if they were real numbers. For 
example, it follows from the results of Problem 6 that 

c 13 /c 2 ~ c 13 ' c 12 = ?10 1 

since c 12 is the multiplicative inverse of c 2 . The fol- 
lowing table will ejiable us to multiply and divide columns^ 
4 3 

modulo x * x +1 quite easily. 

. * m - * TABLE 2 

Column Multiplication for the Matrix H« of Section 4 



(c 2 ) 2 


= c 4 


(c 2 ) ~ c 7 


v 12 


= c 3 


(7) (c 2 ) 3 


" c 8 


(C 2 ) " c 14 / . 


(c 2 ) 


" C 6 


(c 2 ) 4 




(c 2 ) - c 5 


(c 2 ) 


" C l? 


(c 2 ) 5 


= C ll 


(c 2 ) = c 1Q 


(c 2 ) 


" C l. 


, ^ 6 


* 

* C 15- 


• ♦ l c 2> . a c 13 







*' ^ f 

Problem 7. Verify that (c 2 ) a c^ and' determine the 
multiplicative inverse of each , 1 £ i £'15. 

By. now you have probably .guessed 1 that choosing h 

pOO = x 4 + x 3 + 1 to get multiplicative closure was no fi , 

accident. It t wasnjt^ ,a discussion* o.f the properties of. 

4 3 ' - * 

x •+ x +1 whpxh guarantee such 'a nice column algebra can 



be found in The Theory of Error-Correcting Codes I %y 
MacWilliams and Sloane. 

The requirement f(c_) = (c ) 2 is legitimate now that - 
we have invented a way to square columns, but it is not 
useful. The. second e^uatioa in (.4) becomes 

2- 2 ' V 

( Ci ) 2 ♦ ( Cj ) 2 = s 2 ,< 

Which is redundant, since 

: (s i )2 " ( c i + c j-> 2 • ' ■ - . 

,1." (c ifc 2 * c j'- c i. + c i- c j + ( c j) 2 • \ 

2 ' ' 1' 

l^j) ^ ( c i"Cj + ?i" c j>^ ( c j) ,( Reme °ber, 1+1-0.) 

2 * ,<^2 



% ••Since s^^ f 0, 

s 2 /s l * <V c j + C^) 2 , 
so that " ^| 

V c j = ( s i> 2 + W 

which implies 



\ 



= s 2' 



That is, the second equation is the square of the first. 

Pressing on,' we try f(c_) = (c ) 3 . The associated 

#hMi m 

system is " 



(8) 



. c i + c j * s l« 



Cc-) 3 (c.) 3 = s 2; 



otice that 



4f - - ' 
>i - (Cj) 3 ♦ (C.) 3 . * 

\ (Cj ♦ Cj)}^)- 2 ♦ c t • . Cj .( Cj > 2 ) 
- Sj • (c^Cj ♦ (cv) 2 ♦ ( Cj ) 2 ) 
= s x • (c.. Cj ♦ (c. * Cj ) 2 ) 



■( 



= s 



1 > (V c j + ( s i) 2 )- 



13 



r 



Substituting for Cj in Cj + Cj = s^* yields 
2 



c. + 



(s x ) ♦ s 2 / Sl 



5 i 



c i . 



which is equivalent to 

(c,) 2 ♦ Sl .c. + (( Sl ) 2 * S^J 

# - Similarly, 



(Cj) ^^ l Cj * ^ l)2 + = °" 

Thus. the columns c i and c^ assojciated with errors in the 
ith and jth positions must satisfy the quadratic equation 

z 2 ♦ Sl ,l + .{( Sl> ) 2 ♦ s 2 ^) - 0.' ; " ' 

If only one error occurs, say in the ith position, then 
c. -^sj and ( Sl ) 3 = * ' ' v \ 

ft - * ' * ^ ' - * 

We have. solved the problem posed in Section 2. The 
code, C 3 , is the set of solutic^ to H 3 m t = 0,- where 



1 ■ 



14 





C 2 








C 15 


f' ->3 

(c x ) . 


(c 2 ) 






• 


^\ 


"0 0. 0 


0 0 


0 


0 


1 


1 1 1 ,1 


f\ f\ f\ 

0 0 0 


1 1 


1 


1 


0 


0 0 0 1 


0 1 1 


0 0 


\ 


1 


0 


0 11Q 


1 0 1 


p.,1 


0 


1 


0 


10 10 



0 1 1 1 0 0 1^0 0 0 0 0 1- o' 1 

001:0111 6- oo *ooi-o 
a o i i i o i o .i o o i o o \ 
j\ o 1. 1 i 1 i i \ V i i o i- o 



Problem 8. Determine the number of codewords in C 3 . 

' Our instructions to the receiver are*: upon receipt 
of r, first compute ' 



Then: m * ' \ ' 

(i>) If ^ » s 2'~ °» assume no errors occurred. ■ * . 

' (ii) Ifs^ 0 -and s 2 ■ (s 1 )' 3 , assume a single error 
,occurred^ in the ith position where s, * c-. 

(.iii) If s : f 0 an'd s 2 . f (s 1 ) 3 , examine 
{c^: 1 < k < 15} for .solutions to .the quadratic equation 

^ & * I 

^ ' 2 * i. .2 ..).■■ 



W - 2 ♦ s^zj t {( Sl ) 2 ♦ s 2 / Sl ) =• 

If two solutions, ^ "and c . , .are jfound, 



assume errors 

occurred in the ith and *j th positions . (Jtherwise, assume, 
three errors occur re'd (and request retransmission:). 

(iv) If s 1 =.Q«and s 2 f 0, assume three errors ' r 
occurred (and request retransmission). 



Example 2 : Suppose r * 101110000110001. Computing 



H 3 r L yields 



s l = c l + c 3 + c 4 + * c 5 + c 10 + c ll + c 15 



= c 



13 



and 



.3 ■ ^ ,3 



t r\ 

3 



, S 2 J < c l> + / c 3^ + ^ + (c 5 ) J ♦ (c 10 ) a - (q 15 ) 



= c io- 



Since Sj f Q and (s^)^ = c^ 7*'s 2 , we are^iri case (iii) 
above. • Equation (9) becomes 

2 * ' , 

2 + c i3 * 2 + c ii - = °? 



11 



since 



* \< s l) + (s 7 )/ Si ;= {c 



2£ 



, ,l0 /r ,11 
(c ? ) /(c ? ) 



22 



■ (c 7 ) , + (c 7 ) 



r (c 2 ) 7 ^-(c 2 J 14 



r c ? .+ c 



Testing for roots, we find that 



( c 3> ,+ c .i3 * c 3 ** c Il = '-°" 



and 



which. imply errors occurred 'in the 3rd a^nd 14th positions 



The message was (probably) 

ip ■ 100110000110011. 



Problem 9. Apply the receiver • s, instructions to 
• r x * 11010O010110010 ■ 

r 2 = oiooooiiio^eebo ' 

r 3 = 110100011000010 ' 

■ * ♦ 

' r 4 * 110000010100011 « 
and recover the corresponding message when possible. 

problem 10..\ Conjecture the form of a parity-check 
matrix, H, for a cade that will detect and correct up to ■* 
three errors in transmission. ' ! 

6. PROBLEM .SOLUTIONS 

1. .r * m + e implies m * r + e * 

m - 101010101010101* , ' - 

* • .+ - - * 

r » 101010111010100 



e m 000000010000001 

c 1 * ^ooooooooooboooK 
c 2 « onoiioiiioioii 

c 3 » 111111111111000 
» 1U000000000101 



3. , H.r C 



0 

1 



The onXy t?#o, coltimns of which sum to 



are the 



' X3th aijtd 15th. Thus c - 000000000000101 and m « r + 
, llllllliillOlOl. ' 0 % ® 



^4. If the ich column is a column of- zeros 'then a single error in 
Che ich column will noc bedececced; i.e., 

Hr * Hm C -f^He C - 0 + 0 = 0 if e ■ 0 0 1 0 0. 

ich posicion 



5. If a single error occurred ic was in che Ach posicion since 
fl 2 r c - c^. Therefore m ■ 111000101001101. Since 
C ll- + C 15 = C l* c 7 = c 4 err °rs in che llch and 15ch posic^ons 
* or in che 3rd arid 7ch posicions give che same syndrome. * 



6. c 1 1 is che mulciplicacive idencicy 

:.3 t .2 



C 12 ' C 



t - c x since c 12 x + x * f ^ ^ x> and 



(x J + xV- x + x 5 J 



■ 4 3 

x 4 + x° + 1 



x' 4 + x 3 ♦ 
* 4 .*Vi 1 



u 13 c 12 g l0 Slnc . e . c i3 1. 0 4 

c 12 ~ x * + x ^ * nd ^ + x * + «<* 3 + **) x$ + x 4 %tx 3 + x 2 , 
- 3 , * ^ * » 

= x + x c : " * 

10 



2 ' 

X + X 



x -f x 4* 1 



6 4 12 

x < + x + x J + x^ " • 

xA + x 5 - 

> x 5 + x*%x 3 ' ' 

* ± x . -fx 

3 

X + X 



5-5*1 
7 v (c 2 0 x J = x J + x + 1 c n 



/ 
. 1* 



& x ± x .+ x * 

A 

* X + x 

x 4 + x 3 + 1 

x 3 + X + 1 



C l \ c l " c 4 * C 6 ' c 8 " C 3 ~ C 9 ' C 13 " C ll * C 10 " C 15 / C 5 ' 

3 



C 7 ' C 14 ' C 2 ' C 12 = l - 



,*8. Since the row. rank of is 8 the dimension of is 15 - 8 = 
Therefore |cJ »'2 7 . 




9. "Compute Hr^ « 



** r l ! S l "* ^' a t m °5 ^ m P^^ es that at least three errors occurred. 
Ask for a retransmission. 




3 

r 2 : ?j) m c 15* s 2 M °* ^ s l^ ' s 2 im P lies |f hat errors occurred 
2nd and- 13th positions since the roots of 



es\fh 



fc 2 2 

*z + c^^ Vz+ ^ c i5^ 3 z + • z + c^ » 0 are c^and-"*?^ . 

nu « OOOOOOtllOlOlOO. i * I 

I ' 3 

r 3 : 8 1 * C 8* S 2 * c 5* ^ s l^ * s 2 im PH es that an error 

occurred in the 8th position, m. ■ 110100001000010. ^ 

' 3 9 \ 

r 4 : s l * ^» s 2 * ^ im pl ies that m^ = r^' 

10. This is a difficult problem, and a complete solution requires the 

development of the finite field theory behind our choice of 

'43*. 
p(x) ■ x + x +1. Such a development is beyond the scope! of 

this module (see The Theory, of Error Correcting Codes I and II , 

by "Mac Williams and Sloane) . Still, an educated guess is ^possible. 

'Indeed, our discussion* at the beginning of Section 5 suggests that f 

we might add four rows to to correct the third *error. The 



matrix we obtain, say H^would be a 12 * 15 matrix and* the 
syndrome of a received word could be written 



where each is* a binary four-component column vector. Thus, 



"1 , "2 

1 y 

C l C 2 



'15 



"15 



g(c 1 ) g(c 2 ) ^ • J- g(c 15 ) 



and 



(*) 



C i + *J + . C k =S 1 

C i 3 , + - C j 3 " s 2 

gCc^) + g( Cj ) + gCc^^ s 3 , 



where errors occur in the ith, jth,and kth positions, and the 

faction g is* to be determined such that (*) ,can be solved 

«*ir •• * * 

uniquely for c , c and c . How should we define g? Since 

£(z ± ) ■ c ± , g(c i ) » c ± and<g(c i ) ■ c ± won't do, let's try 
8(c i ) * t ± m Then 



4 4 4 
s 3 = c i + c j + . c k 

. '_ , 2.2 . , 2.2 • 2.2 
- (c t ) + (Cj ) + (c k > 

*2 2 2 2 2 

y - 2 2 2 2 

= off ^ 

2 T 2 



That is, the first and- third equations in (*) are redundant 
Our riext^ guess is gCc^) =« c 5 . Hard as you may try, you won ! t 
be 1 able^o find any redundancy in £he system. 



3 3 3 



Conjectured 



C l C 2 



~15 



3 3 

1 J 2 
~5 ~5 

C i C 2 



u 15 



"15 



1. MODEL EXAM 

The binary matrix 

11110 
0 1111 
10 10* 1_ 

determines a code contained in Z-^. 

a. Compute the syndrome of r = 10111. 

b. Is r a codeword? 

* 

c. Hoy many codewords are there in C? 

d. List all of the codeword *in C , 

e. If r is received, what is, the most likely message 

f. The code C can detect all single errors. Can it 
correct all single errors? Why? 

Solve the equation x 6 +,x 4 + x 2 + 1 = p(x) (modulo 
x 4 + x 3 + 1) , where 4 pi 
degree at most three; 



4 3 

x + x +1), where p(x) is a binary polynomial of 



This problem refers to the code C^tof Section 5. 
Determine,* if possible, the message if 
r - 100100110010110 is deceived. 

is it possible to apply the results of the module 
to*a binary symmetric channel igr which- p*> Jj? 



s 



8. EXAM SOLUTIONS ^ 



1. a. 1 
1 
1_ 

, —in. _ 

,< * ' * 



5-3 2 



1^1 1 1*0 

i o r o i 



m l 




"o; 






0 






0 


m 4 




m 5 





implies^m^ = *m^, m^ 35 + m^, and 

The Jrfe fore, fc ^ N v 

G k {00000, 01010^ 11001, 10011 \. 



^ 10011. 



f.. No. If 00010 'Is reaelved, then 00000 and 'OIOIO are equally 
likely to have* been sent. 

: 'J ' * 'i' 

2. p(x) * x + 1, since 



+ x " 



x^ + x 3 + 1 



" 4 

+ X 



x 6 + x 5 



5 J. 4 

X + X 
X + X 



+ X * + 1 ' 



+ 1 



+ X 



x + a. 



.0' 

o' 
1 

.1 

0 
0 
0 



0 implies *;that at least one*error occurred in * 
transmission . \ 



S4 



23 



Since 



- c^, and = 



w 8 ^2 ' 



we conclude that a single error occurred, in the second position. 
That is, the 'message .Was 1101O01100l6ll0. * 



Yes. If a 0'is received, rewrite it 



as a 1. If a 1 is received, 



rewrite ,it as a 0. 
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.STUDENT FORM 1 
Request for. Help 



Return to: 
EDC/UMAP 
*55 Chapel St. 
Newton, MA 02160 



Student: If you have troj^e with a specific part of this unit, please fill 
out this form and take it to your Instructor for assistance. The information 
you give will help the author to revise the trait. 

Your Name 



Page > 








O Upper 


' OR 


Section 


OR 


OMiddle 




Paragraph 




O Lower 






3 



Unit No. 



Description of Difficulty: (Please be specific) 



Model Exam 
Problem No„_ 

Text * 
Problem No; 



Instructor : Please indicate your ^resolut ion of the difficulty in this box. 
£^ "Corrected errors in materials. List corrections here: 



\ 



if* 
V 



J 



( J .Gave student better explanation, example, or procedure than in unit. 
Give brief outline of your addition here: 



f J Assisted student in acquiring general learning and problem-solving 
skills (not using examples from this unit.)' 



c 1 



Instructor's Signatute_ 



ERIC" 



Please use reverse if ^necessary. 



Return to: 

STUDENT FORM 2' EDC/UMAP ' ¥ 

it j*. rx 55 Chapel St* 

^-Upit Questionnaire- „ . r WA A «.*„ 

• vTS Newton, MA 02160 



Name - Unit No* ' Date 

. 3 ; 5 ; 5i — 

Institution . Course No. * 



Check the choice for each question that comes closest to your personal opinion. 

V V 

I. How useful was t'he amount \of detail in the unit ? 

" Not- enough detail' to understand the unit • ^ 



JtJnit would have been clearer with more detail 
Appropriate amount of detail 



JUnit was bccasionally 'too detailed, but this 'was, not distracting 



Too much detail; I waa\ often distracted 

• ' . * ) 

2. How helpful were the problem answers ' 

Sample solutions .were too brief; I could not do 'the intermediate st/eps' 

: Sufficient information was given to solve the problems 
* Sample solutions were too detailed; I didn't need them 




3. Except for fulfilling the prerequisites, how much did you use other sources (for 
example, instructor, friends, or other books) in order to understand the unit? 

A Lot ^Somewhat A Little * Not at all 

4* How long was this unit in comparison to the amount of time you generally spend on 
a lesson (lecture and homework assignment) in a typical math or science course? 

' -Much ;5 Somewhat * ; * About Somewhat Much 
Longed' Longer the Same Shorter Shorter 



We^e any of \the following parts oj? the unit confysing or distracting ? (Check 
as many as a^pply.) "™ ~ 

Prerequisites . . 

Statement of skills and concepts ('objectives) x 

2 Paragraph headings * 



^Examples ; l > 

^Special Assistance Supplement (if present) 
Other, please explain 



6'. Were any of the following parts of the unit particularly helpful? (Check as many 
as apply . ) 

Prerequisites 

Statement.' of skills and toncepts (objectives) 



^Examples 
_Problems 

^Paragraph headings 
Table of Contents , 



^ Special Assistance Supplement ; (if present) m 

Other , please^&cplain^ . 



Please describe anything in- the unit* that you did not particularly like. 

Please describe anything that you found particylarly\ helpful. (Please use the back of 

this sheet if ' you need, more space.) M 

***** 



